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OF AN OPTICAL DEVICE 

CROSS REFERENCE TO RELATED APPLICATIONS 
[0001] This invention claims priority of German Patent Application DE 101 06 698.8, the 
entirety of which is incorporated herein by reference. 

BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

[0002] The invention concerns a method and an apparatus for automatically focusing an 
optical device such as, for example, a surgical stereomicroscope, with which it is possible to 

q focus on the specimen plane which requires the greatest visual acuity. 

Q 

M 2. Description of the Related Art 

yj 

M [0003] In many applications, especially clinical ones, work is performed in only a small 

N 

y, region of a field of view. Moreover, the image content in the small environment in which 

!L- work is being performed exhibits the greatest changes as a result of movement of the 

Cj 

fU instrument on which the work is being performed. Therefore, as a viewer needs the greatest 

[jj focal sharpness at precisely where work is being performed, focusing should be performed 

^3 specifically in this region. 

[0004] In addition to manual adjustment of the focal plane of an optical device, there are 
many^ automatic s yste ms that perform^a fo.c al ad jugtme nt based on a measuremen ^signal §j$Af ftC^ 
which, in turn, is generated substantially based on a measurement of th e distance to the (r 
specimen. If the X and Y coordinate axes lie in a plane common to specimen plane, and the 
Z coordinate axis extends parallel to the optical axis (and perpendicular to the X and Y axes), 
the X and Y coordinates must be defined in order to ascertain a distance from the optical 
system to a point in the specimen plane. In simple systems, these coordinates are defined as 
Xo = Y 0 = 0. In more complex systems, the coordinates to be defined (X 0 , Y 0 ) are designated 
by means of a joystick. Complex systems can ascertain, for example by "eye tracking," the 
coordinates (Xo, Yo) of a point being viewed by the observer's eye. In addition, the complex 
measurement system can ascertain the vertical spacing between that point (Xo, Yo) and the 
objective, i.e. Z 0 . DE-A-1 928 432, Commissariat a l'Energie Atomique, Paris, which is 
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incorporated herein by reference, describes adapting the Z 0 coordinate, based on the change 
in brightness of the image rays (as the focus setting). 

[0005] In addition, German Documents DE-A-2 053 017 and DE-B-2 102 922, both of which 
are incorporated herein by reference, respectively, describe: (a) compensating for a slit 
image using displaceable mirrors; and (b) an invisible ray bundle with a photoelectric device. 
[0006] The inventor has recognized that the known systems are disadvantageous in the 
following respects: (a) a manual or electric focusing system is slow, and diverts the viewer 
from his or her task or activity; (b) the existing autofocus systems focus on a specific 
preselected region (Xo, Yo) which must be deliberately modified (with manual or electric 
activation); (c) "eye tracking" devices are: (i) technically complex; (ii) require additional 
equipment to track the user's eye; and (iii) limit the field of view of assistants, who are 
observing via a reflected beam path, to the direction of view of the principal observer; and (d) 



0 all known methods ignore the use of instruments by the user, or in some circumstances are 

N 

yj interfered with thereby. 

Q [0007] For these and other reasons, a new apparatus and method are needed for automatically 

^ focusing a target region of a specimen. 

Q SUMMARY OF THE INVENTION 

nj 

Tj; [0008] A first aspect of the invention involves a method for auto focusing the objective of an 

W optical device onto a target region of a specimen. The method includes: (a) positioning the 

Q 

pj specimen defining a plurality of regions in an image field of the optical device; (b) 

determining the frequency movements for each of the plurality of regions of the specimen; (c) 
defining as the target region the region among the plurality of regions of the specimen which 
exhibits a predetermined frequency movements value; (d) determining the vertical distance 
Zi between the objective and the target region; (e) generating a first signal corresponding to 
the calculated distance Z\' 9 (f) inputting the first signal into an autofocus system unit 
connected to a focusing device; (g) activating the focusing device, in response to the first 
signal; and (h) focusing automatically the objective using the focusing device. 
[0009] The method may also include: (i) analyzing an image corresponding to the specimen 
for the evaluation of a first optically detectable specimen-related change. Moreover, this 
change may be categorized such as, for example, into categories corresponding to fast and 
slow changes, changes in brightness, and occurrence of reflections. Moreover, multiple 
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changes may be analyzed and categorized. 

[0010] The target region in the aforementioned method may correspond to a region in the 
specimen which exhibits the greatest frequency movements per unit time. Similarly, if 
multiple changes are analyzed, one of the regions corresponding to the changes can be 
defined as the target region. 

[0011] The method may also include, between steps (f) and (g): (f)(1) determining the 
vertical distance between the objective and a region on the specimen on which the objective 
is focused, said distance defining a second vertical distance; (f)(2) generating a second signal 
corresponding to the second vertical distance; (f)(3) inputting the second signal 
corresponding to the second vertical distance into the autofocus system unit; and (f)(4) 
comparing the second signal to the first signal to determine whether the second vertical 
distance is equal to the calculated vertical distance Z\. In addition, if the calculated vertical 
distance Z\ and the second vertical distance are not equal, the method may further include: 
(f)(5) subtracting the calculated vertical distance Z\ from the second vertical distance to 
define a difference. In addition, if the difference is positive, the step (h) may include 
increasing the distance between the specimen and the objective. Similarly, if the difference is 
negative, the step (h) may include decreasing the distance between the specimen and the 
objective. 

[0012] The aforementioned method may be adapted to include (i) identifying objects moving 
within the image field. It should also be noted that the step (b) may include the steps of: 
(b)(1) resolving each of the plurality of regions of the specimen into a corresponding pixel 
having an X and Y coordinate; (b)(2) generating, using an image sensor, first electrical 
signals corresponding to each of the pixels; and (b)(3) using the first electrical signals 
corresponding to the pixels to generate a first image representation of the specimen. 
Moreover, the first electrical signals corresponding to the pixels may be video signals. In 
addition, step (b) may also include: (b)(4) generating, using the image sensor, second 
electrical signals corresponding to each of the pixels; (b)(5) using the second electrical 
signals corresponding to the pixels to generate a second image representation of the 
specimen; and (b)(6) comparing, for each pixel, the second electrical signal to the first 
electrical signal to define the frequency movements of the region of the specimen 
corresponding to the pixel. Further, step (b) may additionally include: (b)(7) displaying the 
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frequency movements of each of the pixels in a visual output which may be a three- 
dimensional histogram having X, Y, and Z coordinates which correspond to the X and Y 
coordinates of the pixels, respectively, and wherein the Z coordinate of the histogram 
corresponds to the frequency movements at each of the pixels. In addition, the pixel having 
the greatest frequency movements may be set as the target region. 

[0013] The invention also addresses an apparatus for automatically focusing an optical 
system. The apparatus includes an image sensor analysis unit; an autofocus system unit; a 
focusing device; and a displaceable objective. In this apparatus, the objective is adapted to be 
automatically moved vertically in response to a signal corresponding to a vertical height 
between a target region of a specimen and a reference point. The image sensor analysis unit 
is adapted to send a signal to the autofocus system unit corresponding to the vertical height of 
the target region. The autofocus system unit is adapted to measure a focus height. The 
autofocus system unit is adapted to compare the focus height to the vertical height between 
the target region and the reference point and yield an output corresponding to said 
comparison. In addition, the focusing device is adapted to move the objective vertically in 
response to the output of the autofocus system unit. 

[0014] In the previously described apparatus, the image sensor analysis unit may be a 
computer. Moreover, if the image sensor is a computer it may be adapted to detect changes 
in electrical signals corresponding to distinct regions of the specimen and the computer may 
be further adapted to analyze the changes. 

[0015] These and other features, aspects, and advantages of the present invention will 
become more apparent from the following description, appended claims, and accompanying 
exemplary embodiments shown in the drawings. 



[0016] The accompanying drawings, which are incorporated in and constitute a part of the 
specification, illustrate embodiments of the invention and together with the description, serve 
to explain the principles of the invention. 

[0017] FIG. 1 shows an exemplary configuration of an image sensor analysis unit according 
to the present invention and an image sensor of a camera having an imaging optical system; 
[0018] FIG. 2 shows a specimen as it appears upon looking through the eyepiece of a 
microscope, the specimen having a pointing instrument in contact therewith; 



BRIEF DESCRIPTION OF THE DRAWINGS 
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[0019] FIG. 3 shows the evaluation of the frequency movements of the instrument tip in the 
form of a histogram; 

[0020] FIG. 4 shows a representation of the histogram of FIG. 3 in two-dimensional form; 
[0021] FIG. 5 shows the frequency of changes over X-Y coordinates of the specimen; 
[0022] FIG. 6 shows a configuration in which two instruments are used as pointing 
instruments; and 

[0023] FIG. 7 explains the geometry of the specimen coordinates. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0024] Reference will now be made in detail to the present preferred embodiments of the 
invention, examples of which are illustrated in the drawings. Wherever possible, the same 
reference numbers will be used throughout the drawings to refer to the same or like parts. 
[0025] FIG. 1 shows an exemplary configuration of an image sensor analysis unit 8 

according to the present invention. Preferably, the image sensor analysis unit 8 is a 

Sj 

y j computer. Also shown in FIG. 1 is an image sensor 1 which may be, for example, a CCD of 

a camera. Further, the image sensor 1 includes an imaging optical system 2, such as a lens, 
M* that acts in conjunction with a beam splitter 3. 

q [0026] The beam splitter 3 is positioned so that a portion of a ray bundle 20 emitted by a 

JjjjJ specimen 10 having a non-planar surface is directed to pass therethrough along a 

Ly transmission axis 11. Between the beam splitter 3 and the specimen 10, there is positioned a 

pi principal objective 5 of an optical device which may be, for example, a microscope. After 

the portion of the ray bundle 20 passes through the principal objective 5 and enters to the 
beam splitter 3, the beam splitter 3 divides the ray bundle portion 20 into two separate beams. 
A first of the beams is directed so that some of the rays from the specimen 10 may go to one 
receptacle which, if the optical device is a microscope, is preferably an eyepiece. The second 
of the beams is directed to the image sensor 1 via a focal axis 12. 

[0027] The image sensor 1 creates an electronically resolved image of the specimen 10 and 
transmits it, by means of a first signal la (e.g., video signal), to the image sensor analysis unit 
8 for further processing. The image may be defined by a plurality of pixels each of which 
corresponds to a particular coordinate in the specimen 10. Moreover, each of the first signals 
la may correspond to a particular coordinate. These first signals la are converted in the 
image sensor analysis unit 8 into second signals 8a (the details of which are hereafter 
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described) which are sent to an autofocus system unit 9. Using the second signals 8a in 
conjunction with distance signals 6a and 6b received from a central point 6 in a plane 5a of 
the principal objective 5, the autofocus system unit 9 activates a focusing device 7 and 
regulates the position of the principal objective 5, as later described in detail. 

[0028] The image sensor anal ysis unit 8 jnay^^by means of suitable programs^ jnternally 

•f— ~ ■ ' 

represent^th e number of image ch anges(alsojgf erred to as " fr equency m ovgi^nts'^^jl^ 
partic ular X-Y coordin a te in the specimen 10 cor res pon ding to the nu mb er of times an 
instruments tip contacts-that_ coordinate. as late r explained. The frequency movements are 
recorded at specific image points on the specimen 10 in the form of a histogram 15 (FIG. 3). 
The X-Y plane of the three-dimensional histogram 15 corresponds to the specimen's X and Y 
coordinates. The Z axis of the histogram 15 represents the frequency movements at a specific 
point in the X-Y plane. According to the present invention, a specific target region 10a of the 
histogram, for example the maximum, is ascertained computationally and the X and Y 



C3 
N 

yj coordinates (Xi, Yi) associated therewith are determined 

M 
N 



[0029] The X and Y coordinates (Xi, Yi) of the target region 10a are sent to ain autofocus 
^ system unit 9 by way ofjh e^secon d signal 8a. The central point 6 of the plane 5a of the 

s ■ ■ "~" ~ " 

p principal objective 5 (i.e., the origin of a measurement beam) is defined to be (0, 0, 0). The 

Vl autofocus system unit 9 determines, for example, by way of the measurement beam (shown in 

W FIGS. 1 and 7), the distance 6c between the determined point (Xi, Yi, Zi) and the central 

ffj point 6. The effective vertical distance Z\ between the determined specimen point (Xi, Yi, 

Zi) and the plane 5a of principal objective 5 can be determined by simple triangulation, as 
explained below. Next, the autofocus system unit 9 similarly determines the vertical distance 
between a point (X', Y', Z') on which the optical device is focused and the determined point 
(Xi, Yi, Zi). On the basis of this determination, the principal objective 5 can be vertically 
moved by means of focusing unit 7 into that position in which the determined specimen point 
(Xj, Yi, Zi) is sharply imaged for the viewer. 

[0030] FIG. 7 explains the geometry of the specimen coordinates. The determined specimen 
point 10a possesses a coordinate (X 1? Yi, Zi) in the specimen space. The point (0, 0, 0) is 
defined as the central point 6 in the reference plane 5a of the principal objective 5 (i.e., the 
point from which the measurement beam 6c originates). The X and Y coordinates (Xi and 
Yi) are ascertained on the basis of the previously described image analysis. The effective 
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* 

distance Z\ to the reference plane 5a of principal objective 5 can be ascertained by means of 
distance measurement, e.g. by simple triangulation, as hereafter explained. 

q 2 = Xi 2 +Y, 2 
Zi = square root(6c 2 - q 2 ) 
Zi = square root(6c 2 -(X! 2 +Yi 2 )) 
[0031] The invention is not limited, however, to a method with a measurement beam 6c. The 
definitive determination of the focal distance Z\ to the corresponding calculated specimen 
point (Xi, Yi, Zi) can also be made with other conventional methods, for example split 
image, edge definition analysis, etc. For example, the height Z\ could also be determined 
with respect to an angle 30 which is defined between a line normal to the plane of the 
specimen 10 and a line from the principal objective 5 to the calculated point (Xi, Yi, Zj). As 
Zi is parallel to the normal line, the angle 30 is shown in two places in Figure 7 which are 
Q equiangular. Similarly, due to the parallel nature of the plane 5a of the principal objective 5 

yj and the plane on which the specimen 10 is located, the distance q is shown in two places. 

Therefore, Z\ can be calculated as follows: 
** q 2 = Xi 2 +Y, 2 

Q q = square root (Xi 2 +Y] 2 ) 

L» tan(angle 30) = q/Zi 

W Z, =q/(tan(angle30)) 

|1J Zi = square root(Xi +Yi )/(tan(angle 30)) 

[0032] After the height Z\ is calculated (and a signal corresponding thereto is sent to the 
autofocus system unit 9), the vertical distance between the principal objective 5 and the 
region 10a on the specimen 10 which is in focus is determined, thereby defining a second 
vertical distance. A signal 6b corresponding to the vertical distance between the principal 
objective 5 and the region in focus is then sent to the autofocus system unit 9. The autofocus 
system unit determines the difference between the signal corresponding to Z\ and the signal 
6b corresponding to the distance between the principal objective 5 and the region in focus, 
the former being subtracted from the latter. If the difference is positive, the autofocus system 
unit 9 instructs (via a signal 7a) the focusing device 7 to increase the distance between the 
specimen 10 and the principal objective 5, this usually occurs by lowering a stage on which 
the specimen 10 exists; the specific increase in distance will depend on the magnitude of the 
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difference. Similarly, if the difference is negative, the auto focus system unit 9 instructs (via a 
signal 7a) the focusing device 7 to decrease the distance between the specimen 10 and the 
principal objective 5, this usually occurs by raising a stage on which the specimen 10 exists; 
the specific decrease in distance will again depend on the magnitude of the difference. 
[0033] FIG. 2 shows the image of a specimen 10 as it appears, for example, in a view field 13 
upon looking through the eyepiece of a microscope. A similar representation of the specimen 
10 will also be made on image sensor 1. A^rgic^ Linstrunxo nJ^l^p ^forms activi ties on 
specimen 10 that result in a change in the position of the instrument's tip; this movement^ 
sequence is depicted symbolically by the line^_5j0^druwn ji^he specimen 10. FIG. 3 shows 



a 3 -dimensional evaluattori "of the frequency movements of the instrument's tip in the form of 
a histogram 15. The X-Y coordinates of the histogram 15 correspond to the specimen's X-Y 
q coordinates; the Z axis of the histogram corresponds to the frequency movements in the 

Q respective image field. Similarly, FIG. 4 shows a two-dimensional evaluation of the 

IM histogram 15 of FIG. 3 in which the number of times the instrument's tip contacts a particular 

l^j X-Y coordinate in the specimen 10 is represented by showing the numeral corresponding to 

that number of contact in that X-Y coordinate. For example, as the coordinate (X=-2, Y=2) 

5 . 

Q in FIG. 2 was contacted by the instrument's tip one time, the numeral "1" appears in that X-Y 

jfU 

y: coordinate. By way of a further example, as the coordinate (X=4, Y=4) was contacted four 

W times, the numeral "4" appears in that X-Y coordinate. 

ilJ [0034] FIG. 5 shows the frequency movements in the X-Y plan over the specimen 10; the 
shape of the pixels is substantially the same as the two-dimensional depiction in FIG. 4. 
Rjggionsex periencin g hi gher frequericy-_mav whic h the 

instrument's tip pass at a hi gher frequency) are depicted as darker histogram pixels. By way 
of example, with respect to FIG. 5, regions through which the instrument's tip passes one 
time are represented by pixels having a line travelling in one direction therethrough (i.e., a 
"/"). Regions through which the tip passes two times have an "X" therein. Finally, regions 
through which the tip passes four times are represented by a grid of lines. It should be 
recognized that the pixels in which a mo vement is indicated correspond directly to the path of 
v t he instrument's tip shown in FIGS. 2-4. 

[0035] FIG. 6 shows a configuration in which two instruments 14a and 14b are used as 
pointing instruments. Also depicted is the relative movement of the two instruments 14a, 
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14b, as shown by the pixel coloration scheme according to FIG. 5. As a result of the dual 
movements 15a, 15b, the invention can be adapted to focus on two separate points (X 5 , Y'), 
(X", Y"), at the choosing of the operator. The vertical spacing between the specimen 10 and 
the principal objective 5 can be changed to correspond to one or both of these separate points. 
Moreover, at the choosing of the operator, the vertical spacing could be changed so that both 
points are equally out of focus, i.e., so that the plane in the specimen 10 which is vertically 
equidistant between the two points would be in focus. 

[0036] The apparatus previously described can be used to identify items moving within the 
image field 13 and/or to analyze images corresponding to the specimen 10. Moreover, by 
analyzing distinct regions of the specimen 10 in various images corresponding thereto, the 
apparatus can detect and categorize specimen-related changes such as, for example, fast and 
q slow changes, brightness and/or occurrence of reflections. In addition, if multiple changes 

are detected, the apparatus can weight the changes to determine which change should be the 

N 

Ul target region. 

[0037] Although the method and apparatus are particularly applicable to a surgeon and to a 
^ surgical microscope and a surgical field, the invention is not limited thereto, but rather is also 

C3 open to other users of optical devices (e.g., video and still cameras, monocular as well as 

p binocular applications). In addition, it will be apparent to those skilled in the art that various 

WJ modifications and variations can be made to the disclosed preferred embodiments of the 

C3 

jpy present invention without departing from the scope or spirit of the invention. Accordingly, it 

should be understood that the apparatus and method described herein are illustrative only and 
are not limiting upon the scope of the invention, which is indicated by the following claims. 
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